Original article / research
Year :
2020 |
Month :
October
|
Volume :
9 |
Issue :
4 |
Page :
BO05 - BO09 |
Full Version
|
|
Sigma Metrics-Tool for Quality Assurance in Clinical Biochemistry Laboratory
|
Richa Singh, MN Vanitha Gowda 1. Resident, Department of Biochemistry, M.S. Ramaiah Medical College and Hospitals, Bengaluru, Karnataka, India.
2. Professor and Head, Department of Biochemistry, M.S. Ramaiah Medical College and Hospitals, Bengaluru, Karnataka, India.
|
|
Correspondence
Address :
Richa Singh, MN Vanitha Gowda, Vanitha Gowda MN,
MSR Nagar, Bengaluru, Karnataka, India.
E-mail: vanithasukesh@hotmail.com
|
| ABSTRACT | | : Introduction: Sigma metrics is a quality management tool used for process improvement which usually comes into application when there is a measurable outcome in the process. It can play an important role in health care laboratory services as Quality Assurance (QA) of the same, is the need of the hour.
Aim: To gauge the performance of a few biochemical parameters by calculating their sigma metrics on a sigma scale.
Materials and Methods: This retrospective study was undertaken using Quality Control (QC) and External QA Scheme (EQAS) data for 17 biochemical parameters from Biochemistry section, diagnostic laboratory of an M.S. Ramaiah Medical College and Hospital, Bengaluru. Sigma values for these parameters were determined and sigma metrics was evaluated for duration of 13 months.
Results: In level 1 coefficient of variation percentage (CV%), five parameters (ALP, calcium, magnesium, triglycerides and HDL-cholesterol) showed an ideal performance of ≥6 sigma level and in level 2 CV%, eight parameters (total bilirubin, urea, creatinine, albumin, AST, total cholesterol, total protein and phosphorus) showed a sigma of ≥6. Quality Goal Index (QGI) for 11 analytes in level 1 and seven analytes in level 2 was <0.8, indicating imprecision. QGI was in the range of 0.8-1.2 for one analyte in level 1 and two analytes in level 2, indicating a problem of both imprecision and inaccuracy.
Conclusion: Sigma metric analysis can serve as a tool to identify the poor assay performance and to assess the efficiency of processes that are in existence. The health care sector can be immensely benefited by implementation of sigma metrics for QA. |
|
Keywords
: Bias, Laboratory proficiency testing, Quality control, Quality goal index |
|
DOI and Others
: DOI: 10.7860/NJLM/2020/45073:2404
Date of Submission: May 15, 2020
Date of Peer Review: Jun 06, 2020
Date of Acceptance: Jul 20, 2020
Date of Publishing: Oct 01, 2020
b#bAuthor Declaration:b?b
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? NA
• Was informed consent obtained from the subjects involved in the study? NA
• For any images presented appropriate consent has been obtained from the subjects. NA |
|
|
INTRODUCTION |
|
Clinical laboratories play a major role in healthcare system (1). In a Clinical Biochemistry Laboratory (CBL), the measures used to assess the QC are Internal QC (IQC) and EQAS (2). Proper documentation, stability and reliability are some of the key features of laboratory QC (3). IQC is an important part of laboratory quality management whose products can determine the reliability of test results (4). Selection of a proper IQC procedure for implementation is the first essential practice in setting up IQC (5). IQC is interpreted using the standard Westgard rules and is run daily, as per National Accreditation Board for Testing and Calibration Laboratories (NABL) guidelines. IQC keeps an eye continuously on the analytical system to check whether the results are reliable enough to be released or not. Contrarily, EQAS sample, which is supplied by an outside agency, is run once every month and is interpreted using the Z-score (5).
Approximately, two-thirds of important clinical decisions on patient management are based on laboratory test results (6). The entire process of testing in a CBL involves pre-analytical, analytical, and post-analytical phases. As there exists a chance of occurrence of error at any step of testing in a clinical laboratory, reduction of errors and an ongoing improvement in testing is required. To provide accurate and reliable reports within the agreed time, it is important to maintain and follow a proper Quality Management System (QMS) (7). This is what led to the evolution of sigma metrics methodology by Bill Smith back in 1986 (8).
Sigma metrics can quantify the exact number of errors done in the analytical phase by the laboratory that cannot be gauged by running the internal and external QCs (3). By using sigma metrics in the laboratory, the number of errors or defects could be exactly quantified; 1-sigma parallels to 6,90,000 errors or defects per million reports, 2-sigma relates to 3,08,000 defects per million reports, 3-sigma equals 66,800 defects per million reports, 4-sigma to 6,210 defects per million reports, 5-sigma to 230 defects per million reports and 6-sigma parallels to 3.4 defects per million reports (9). Sigma is used in statistics to represent the SD, which indicates the degree of variation in a process. A 5% error rate corresponds to a 3.15 sigma performance and a 1% error rate corresponds to 3.85 sigma (10). Three-sigma is taken as the minimum allowable sigma for routine performance and six-sigma as the goal for world-class quality (11). The present study is among the few that have taken into consideration more than 15 biochemical parameters for studying the effectivity of sigma matrices. This study was conducted to gauge the performance of individual biochemical parameters in MS Ramaiah Hospital Clinical Laboratory and find out the errors associated with each parameter.
|
|
|
Material and Methods |
|
This cross-sectional study was conducted in the Biochemistry Section of the Diagnostic Laboratory of M.S. Ramaiah Medical College and Hospital, Bengaluru from July 2017 to July 2018, using IQC and EQAS data of 17 biochemical parameters selected randomly (urea, creatinine, total bilirubin, aspartate aminotransferase, alanine aminotransferase, albumin, alkaline phosphatase, total cholesterol, total protein, phosphorus, calcium, magnesium, triglycerides, high density lipoproteins, glucose, sodium and potassium). Data analysis was done from July 2017 to July 2018 and the study was conducted from 1-10-18 to 20-10-18 (20 days). The data was obtained for IQC in terms of Coefficient of Variation percent (CV%) and for EQAS in terms of bias% using COBAS 6000 autoanalyser, in the diagnostic laboratory. Sigma values for these parameters were determined and sigma metrics was evaluated. Sigma (s) value is calculated with the following formula (10).
Sigma metrics (s) = TEa%-Bias%/CV%
Where, TEa is total allowable error percentage and CV is coefficient of variation.
TEa= Allowable Bias + 1.65 × Allowable Imprecision
Where, the allowable bias = 0.25 × (CVw2 + CVb2)1/2, the allowable imprecision = 0.5 × CVw, CVb is the inter individual imprecision, and CVw is the intra individual imprecision (12).
Bias% = Our EQAS result- peer group mean (using the same instrument and method) X 100
Peer group mean (using the same instrument and method)
CV was measured based on below formula, using Bio-Rad internal QC1 for all the parameters.
CV% = SD X (100)/Mean
The QGI ratio denotes the relative extent to which both bias and precision meet their respective quality goals (14). The QGI ratio was calculated using the following formula (7):
QGI = Bias X CV%/1.5
TEa values were obtained from the Clinical Laboratory Improvement Amendment (CLIA) (13). The bias percentage for each parameter was calculated using Bio-Rad-EQAS (1).
QGI can be used to assess the reason for lower sigma (due to imprecision or inaccuracy or both) in some analytes (Table/Fig 1). QGI ratio of <0.8 indicated imprecision, ratio of 0.8-1.2 indicated imprecision and inaccuracy and a ratio >1.2 indicated inaccuracy and was used in case test parameters fell short of six-sigma quality.
STATISTICAL ANALYSIS
Microsoft Excel spreadsheet version 2010 was used for statistical analysis. Bias, CV, QGI and sigma metrics were calculated using the above formulae. Bias and CV were presented as percentages.
|
|
|
Results |
|
The sigma metrics and QGI ratio for 13 months (July 2017- July 2018) were calculated using TEa, CV% (level-1 & level-2) and bias% for 17 biochemical parameters as given in [Tables/Fig-2], [Tables/Fig-3],[Tables/Fig-4]. Among the 17 analytes observed in level -1 IQC (Table/Fig 5), five analytes showed an ideal performance of ≥6 sigma level, eight analytes showed sigma level <6, whereas, four analytes showed a level <3, indicating poor performance.
For level-2 IQC in (Table/Fig 5), seven analytes showed a performance of >6 sigma level, six analytes showed a sigma value of ≤6, whereas, four analytes showed a value of <3, indicating poor performance.
From (Table/Fig 5), it was found that QGI for 11 analytes in level-1 and seven analytes in level-2 was <0.8, indicating imprecision. QGI was in the range of 0.8-1.2 for one analyte in level-1 and two analytes in level-2, indicating a problem of both imprecision and inaccuracy.
|
|
|
Discussion |
|
Laboratory performance has been measured as rate of errors or defects that occur per million tests or products. Utilisation of sigma matrices can be useful for measuring the performance of testing processes and service provision. Designing own QC procedures is considered as a good laboratory practice that a laboratory can engage in, to ensure a good quality of reports for the patient (15). Sigma metric analysis can be used to minimise the errors in the laboratory as it is a widely accepted universal benchmark of quality measurement.
In this study, six sigma values were obtained for magnesium, ALP, triglyceride, and HDL cholesterol, in both IQC levels, and were in accordance with previously conducted studies (5),(10),(16). The results of the current study indicated that the methodologies adopted by in-study laboratory for estimation of the above analytes were appropriate and following very stringent QC protocols were not needed for these analytes. Following 13S Westgard rule alone would suffice in these cases.
The sigma values for total protein, creatinine and phosphorus obtained in this study were between 3-6 for both the IQC levels. The values for total protein were found to be 4.36 and 4.67 in levels 1 and 2, respectively. Whereas, in the study done by Kumar BV and Mohan T, the sigma values obtained for total proteins were 3.00 and 3.27 in levels 1 and 2, respectively (5). For the sigma values between 4-6, indicating an acceptable performance a laboratory should follow Westgard Multirules, and QCs should be run twice daily (17). But, for sigma values of 3-4 that indicate poor laboratory performance, Westgard Multirules need to be followed stringently and two levels of QCs should be run twice daily (5).
Authors found a sigma value of <3 and QGI ratio of <0.8 for sodium and potassium for both levels of IQC, indicating a problematic analyte and an imprecision issue, thereby pointing that the daily workflow needs to be revised. For such analytes, the frequency of QC runs has to be increased as per Westgard rules and the root cause of the problem should be found. Additionally, 2 or 4 IQC measurement of both IQC levels is needed with each daily run of 4 or 2. By incorporating the required changes in the workflow, as the sigma increases, consistency, steadiness, reliability, and overall performance of the test improves, followed by a decrease in the operating costs (18). The application of six sigma is hence proved to be helpful in assessing the laboratory testing processes in the study laboratory. However, there seem to be limitations in the clinical application of sigma metrics for some analytes like hormones. For such analytes, CV percentage and bias percentage were found to be better than sigma matrices (12).
Limitation(s)
Reference methods were not used for certain analytes, and external quality control system was used due to financial constraints. However, this study provided certain strong points that helped refine the previously followed QC protocol. The results of this study helped to improve the overall quality of the test procedure and also minimise the excessive QC monitoring, thereby reducing the overall costs for analysis with higher sigma metrices.
Every clinical diagnostic laboratory should design their QC procedure and use six-sigma matrices for QC monitoring to improve the performance of the laboratory. This would make it easy for implementing changes in laboratory consumption of QC materials along with calibrators and reagents. This would also aid in interpretation of control rules and frequency of running QC materials for respective analyte. In addition to sigma metrics, other visual tools such as method decision charts could be used, which transform the sigma metrics into a simple in-built dashboard. It allows a laboratory to get a view of the working of the instruments. In addition, the Operational Specifications chart can be used, which is a graph like the method decision chart, that tells about various rules and controls required to provide the necessary error detection.
|
|
|
Conclusion |
|
Sigma metric analysis can serve as a tool to identify the poor assay performance and to assess the efficiency of processes that are in existence. In this study, the sigma values observed for magnesium, ALP, triglyceride, and HDL cholesterol, in both the levels were >6. A satisfactory sigma (>3) was obtained for creatinine, total protein, and phosphorus, in both the IQC levels. Whereas a sigma value of <3 was observed for sodium and potassium in both levels. The six-sigma purpose is to lessen both variance and QC processes to assure compliance with the critical specifications. The health care sector can be immensely benefited by implementation of sigma metrics for QA.
|
|
| 1. | Thomas V, Desai PB, Mithrason AT. Evaluation of clinical biochemistry laboratory performance using sigma metrics. Int J Clin Biochem Res. 2018;5:604-07.
[ Google Scholar] | 2. | Westgard JO. Internal quality control: planning and implementation strategies. Ann Clin Biochem. 2003;40:593-611.
[ Google Scholar] | 3. | Badrick T. The quality control system. Clin Biochem Rev. 2008;29:S67-S70.
[ Google Scholar] | 4. | Xu G-P, Wu L-F, Li J-J, Gao Q, Liu Z-D, Kang Q-H, et al. Performance Assessment of Internal Quality Control (IQC) products in blood transfusion compatibility testing in China. PLoS ONE. 2015;10:e0141145.
[ Google Scholar] | 5. | Kumar BV, Mohan T. Sigma metrics as a tool for evaluating the performance of internal quality control in a clinical chemistry laboratory. J Lab Physicians. 2018;10(2):194-99.
[ Google Scholar] | 6. | Forsman RW. Why is the laboratory an afterthought for managed care organizations? Clin Chem. 1996;42:813-16.
[ Google Scholar] | 7. | Verma M, Dahiya K, Ghalaut VS, Dhupper V. Assessment of quality control system by sigma metrics and quality goal index ratio: A roadmap towards preparation for NABL. World J Methodol. 2018;8:44-50.
[ Google Scholar] | 8. | Angmo D, Kant S. Six sigma implementation in healthcare industry: Past, present and future. Int J Eng Res Technol. 2015;4:1078-82.
[ Google Scholar] | 9. | Westgard JO, Westgard SA. The quality of laboratory testing today: an assessment of sigma metrics for analytic quality using performance data from proficiency testing surveys and the CLIA criteria for acceptable performance. Am J Clin Pathol. 2006;125:343-54.
[ Google Scholar] | 10. | Lakshman M, Ravindra Reddy B, Bhulaxmi P, Malathi K, Salma M, Prakashan S. Evaluation of sigma metrics in a Medical Biochemistry Lab. International Journal of Biomedical Research. 2015;6:164-71.
[ Google Scholar] | 11. | Patel A, Patel P, Jain S. Evaluating performance of our clinical biochemistry laboratory by application of sigma metrics & other quality indicators-A pilot study. Int J of Res in Pharmacology & Pharmacotherapeutics 2015;4:349-53.
[ Google Scholar] | 12. | Oosterhuis WP. Gross overestimation of total allowable error based on biological variation. Clinical Chemistry. 2011;57:1334-36.
[ Google Scholar] | 13. | Federal Register: Clinical Laboratory Improvement Amendments of 1988 (CLIA) Proficiency Testing Regulations Related to Analytes and Acceptable Performance; Extension of Comment Period [Internet]. 2019 [Cited 2019 Dec 20]. Available from: https://www.federalregister.gov/documents/2019/04/08/2019-06819/clinical-laboratory-improvement-amendments-of-1988-clia-proficiency-testing-regulations-related-to.
[ Google Scholar] | 14. | Westgard JO, Westgard SA. An assessment of s metrics for analytic quality using performance data from proficiency testing surveys and the CLIA criteria for acceptable performance. J Vet Diagn Invest. 2008;20:536-44.
[ Google Scholar] | 15. | ISO-ISO 15189:2012-Medical laboratories — Requirements for quality and competence [Internet]. 2012 [Updated 2014 August; Cited 2019 Dec 20]. Available from: https://www.iso.org/standard/56115.html
[ Google Scholar] | 16. | Singh B, Goswami B, Gupta VK, Chawla R, Mallika V. Application of sigma metrics for the assessment of quality assurance in clinical biochemistry laboratory in India: A Pilot study. Indian J Clin Biochem. 2011;26:131-35.
[ Google Scholar] | 17. | Shah S, Saini R, Singh SB, Aggarwal O, Goel AK. Six sigma metrics and quality control in clinical laboratory. Int J Med Res Rev. 2014;2:140-49.
[ Google Scholar] | 18. | Sarewitz SJ. Evaluating laboratory performance with the six sigma scale. Arch Pathol Lab Med. 2000;124:1748. [ Google Scholar]
|
|
|
TABLES AND FIGURES | |
|
|
|
|
|